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Abstract 



High-inference measures of teacher process variables were taken on 
a sample of 31 teachers selected because of their consistency in producing 
student learning gains on the Metropolitan Achievement Test and were cor- 
relaled with student outcome measures. Correlations showing the streng + h of 
relationships with success in producing student gains are presented. Data 
represent findings from the first year of a two-year study attempting to iso- 
late correlates of effective teaching. 



HIGH- INFERENCE BEHAVIORAL RATINGS AS CORRELATES OF 
TEACHING EFFECTIVENESS 



Carolyn M. Evertson and Jere F. Brophy 
The University of Texas at Austin 

The findings reported in this paper are based on data taken from +he 
first year of a two-year naturalistic study of elementary teachers selected 
because of their consistency across 3 years in producing student learning gains 
on the Metropolitan Achievement Tests. 

Of 275 teachers working at grades two and three, 88 second grade and 
77 third grade teachers who had tai ght at the same grade for 5 years or more 
were identified. Thus the sample included only experienced teachers. Since the 
district administered the Metropolitan Achievement Tesis each fall, it was pos- 
sible to use these data for determining teacher effectiveness. Students 1 re- 
sidual gains on the subtests of the Metropolitan battery were computed by us- 
ing scores from the beginning of the year as pre-score covariates for adjust- 
ing gains between that testing and the following testing a year later at the 
beginning of the following grade. In addition to computing residual gains by 
using pre-scores as covariates, student differences were taken into account by 
separately computing the data for Title I vs. Non-Title I schools, for boys vs. 
girls, and for each of the 3 years separately. Thus, for example, the formula 
used In computing the residual gain scores for a boy in second grade in a Title I 
school In 1969 was based on the distribution of pre- and post-scores for all 
boys In the second grade in Title I schools in 1969. 

Each student took 3 language arts subtests (word knowledge, word discrim- 
ination, and reading) and either I or 2 arithmetic subtests. The number of 
arithmetic subtests available depended upon which Metropolitan battery had been 



used and on how the data were recorded in the school records. For example, 
a given child's arithmetic data might contain an arithmetic computation subtest 
only, a combination score reflecting both computation and reasoning, or 2 separ- 
ate scores, I for computation and I for reasoning. 

Two sets of computations of residual gain scores were made for each 
grade, because of differences In the test batteries used in Title I and Non-Title 
I schools. For each of these 4 data sets, residual gain scores were computed 
for each student within sex and within each of the 3 years on each subtest, using 
the studenl's pre-score as a covariate. These residual gain scores for students 
were then collated by classroom, and a mean residual gain score was computed for 
each teacher for each subtest for each of the 3 years included in the study 
(Brophy, 1972). 

Many teachers showed constancy across subtests within years as well as 
within subtests across years, so that teachers who produced generally consistent 
gains across subtests and across the two sexes could be identified. The 31 
teachers included in the process observation study the first year (19/1-1972) 
were selected from this consistent group. 

The teachers were divided fairlv evenly between second and third grade, 
and the socio-economic status of their students ranged from upper middle to 
lower lower class. Several classes had a high concentration of black and Chi- 
cano students. The teacher data included process measures of classroom behavior 
and personality and attitude data from pencil and paper tests. These process 
measures included both low-inference behavioral observation systems and high- 
Inference rating: Each teacher was observed for 2 mornings and 2 afternoons 
during the spring semester, for a total of about 8 hours. The main low- infer- 
ence coding Instrument used was based on the Brophy-Good Dyadic Interaction 



System (Brophy and Good, 1970). The system allowed for coding such variables 
as teacher vs. student initiation of contacts, types of interactions (academic, 
procedural, or behavioral-disciplinary), difficulty level of teacher questions, 
quality of student responses, quantity and quali+y of teacher feedback and eval- 
uative reactions to student response and student work, and the teacher's method 
and general effectiveness in handling classroom management and disciplinary 
problems. 

A second coding ins+rument (created by project staff member Nancy Moore) 
was used on a subsample of 10 teachers (5 high and 5 low effective) who were 
observed twice during group instruction activities. This instrument was es- 
pecially constructed to measure group instruction methodological variables, such 
as lesson composition, sequence, and clarity, teacher questioning patterns, and 
handling of seatwork assignments (see Brophy and Evertson, 1973, for a detailed 
description and discussion of the findings from these systems). 

Teachers were asked to fill out the pencil and paper attitude and person- 
ality measures included in the COMPASS battery developed by the Research and 
Development Center for Teacher Education (Veldman, 1972). These measures dealt 
with levels of teacher concerns about teaching (Fuller, 1969), the teacher's 
overall self-concept and her concept of herself as a teacher, her coping style, 
other aspects of her personality, and certain demographic data (see Peck and 
Veldman, 1973). 

The high- inference measures to be reported here were chiefly of two types. 
The first was a set of 12 classroom observation scales developed by Emmer (1973) 
from factor-analytic studies of interaction variables common to several of the 
widely used behavioral observation systems. These 5-point rating scales Included 
ratings of student attention, teacher enthusiasm, clarity, positive and negative 
affect, task orlentalion, cognitive level of questions, student passivity, pupil- 



pupil interaction, and percentage of time spent in lectures and demonstrations. 
The scales were marked several times during each of the 4 observations by class- 
room coders who were also using the expanded Brophy-Good System discussed in 
Brophy and Evertson (1973). 

Secondly, following their last 2 visits to each teacher's classroom, 
observers filled out 41 h i gh-i nf erence ratings and 15 high-i nference checklists 
and percentage estimates. Items for these measures were gathered from several 
sources, but most of them dealt with variables which were not covered in the 
low-inference behavioral coding system. Inter-observer agreement on Emmer's 
Classroom Observation Scales ranged from 7256 to 95%, with an average agreement 
of Q3%. The ratings agreement ranged from 5056 in one case to 985?, with an average 
agreement of 1Q%. The raw agreement data are presented in Tables 2 and 4 for the 
checklists and percentage estimates. No actual inter-observer agreement percent- 
ages were computed for these measures because scores would vary depending upon 
the formulas used. It was decided that the raw data give a more accurate picture 
of the real agreement. 

During each classroom observation, coders also noted the amount of time 
allotted to each subject as well as to special activities such as storytelling, 
TV, art, and transitions. The time utilization results are reported in the 
first part cf Table 2. 

For the most part, the h i gh-i nference measures bore out the findings 
from the low-inference behavioral coding systems (Brophy and Everrson, 1973). 
The correlations reveal several interesting findings. First, with few exceptions, 
significant correlations between process variables and gain scores typically 
involved only one or two of the subtests, despite the fact that teachers had 
been selected because of their general consistency in producing student gains 
across all subtests. Thus it appears that certain teacher behaviors are more 



fmportant for student gain !n some subject areas than in others. 

Many variables which correlated significantly with pupil gains in other 
studies did so in this one, but many did not. For example, teacher warmth, cog- 
nitive level of questions, enthusiasm, amount of student talk, peer-tutor i ng, 
solidarity with students (rapport), and patience all failed to show significant 
correlations with gains. There remains the possibility that some of these 
variables are curvi 1 1 near ly related to the criterion, however. Analyses address* 
Ing this question are being completed and will be summarized in a future re- 
port. 

In addition to correlations across the total sample, analyses were also 
performed separately for Title I (low SES) and Non-Title I (high SES) schools. 

Results 

Data will be discussed within two general categories and grouped within 
categories under thre^ ^ad headings: I.) variables which pertain to non-aca- 
demic classroom manage,,, , control, and organization; 2.) variables which can 
be considered teacher techniques or behaviors in presenting subject matter; 3.) 
student variables which are pupil behaviors not under the direct control of the 
teacher. 

The first category will include variables which are correlated signifi- 
cantly in the total sample and, either significantly or In the same direction, 
In each of the two subgroups (Title I and Non-Title I). 

Correlations for the Total Group 

A. Classroom management and control (positive relations! „s) 

1. Room appearance is attractive and uncrowded. 

2. Students are expected to care for their own needs without getting per- 
mission. 



3. Teacher explains rules or decisions when the reasons for them are not 
obvious, 

4. Teacher is well-organized and we I I -prepared; she is task-oriented and 
doesn't waste time. 

5. Teacher monitors the class regularly, knows what is going on and keeps 
herself aware of events in other parts of the classroom. 

6. Transitions between activities are smooth and efficient with little 
wasted time. 

7. Classroom jobs are determined by some automatic system instead of the 
teacher's spending a great amount of time deciding who is going to do 
particular tasks . 

8. A high percentage of teacher structured time in math (correlates 
with both the reading and arithmetic computation subtests). 

Classroom management and control (negative relationships) 
I. Chaotic, poorly planned class schedules. 

Teacher techniques or methods (positive relationships) 

1. Teacher often addresses problems or questions to the whole class as 
opposed to individuals or subgroups. 

2. Teacher spends a high percentage of time in lectures, answering pupil 
questions, demonstrations, and presenting substantive information to th 
class as opposed to question ng students, giving procedural directions, 
and evaluating (praising and criticizing) student behavior. 

3. Seatwork assignments are appropriate to the task at hand and to the in- 
dividual level of each child. 

4. When a student doesn't understand his seatwork, she asks another child 
to explain rather than allowing Interruptions In what she Is doing, or 
delaying the child In order to explain later. 

5. When helping a child she usually bends close and gets down to his level 
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6. Teacher uses visual aids, pictures, and films-trips as materials *or free 

time activities. 

D. Teaching techniques or methods (negative relationships) 

1. Teacher's allowing Interruptions while she is working with individuals 
or group to explain to a child who doesn't understand. 

2. Teacher's delaying a child who doesn't understand and failing to con- 
tact him later. 

3. Giving overly long, explicit, repetitive directions. (Possibly over- 
dwelling or needlessly rehashing i nformati on, which most of the class 
already knows, serves to "turn off" or lose students.) 

4. Giving encouragement but not specific help when a child is having trouble 
with an assignment. 

5. When a child is stuck during reading group, teacher gives him only 
the Initial sound or syllable of the word. 

6. Having books available in the room (not necessarily used). 

7. A high percentage of teacher structured time. 

E. Student behaviors (positive relationships) 

I. High general levei of student attention and a high percentage of students 
attending when lessons are explained or directions given. 

F. Student behaviors (negative relationships) 

1. Copying from a neighbor rather than asking for help when having trouble 
with an assignment. 

2. When students are not working, they are being distracted by activities 
in other groups going on in the room. 

In the second category are those variables which were significantly 
correlated with student learning only in one group. 

Correlations in Title I 

A. Classroom management and control (positive relationships) 
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I. Keeping child !n after school as punishment. 
Classroom management and control (negotlve relationships) 

1. Underreact! ng to discipline oroblems so that serious problems go un- 
resolved. (This receives some support from the behavioral data iaken 
with the classroom coding sys + em. Thus it seems more important for the 
teacher to stop firmly any control problems before they get out of hand 
than it fs to criticize or punish offenders aft^r the problem or be- 
havior has gotten out of control.) 

2. Allowing 4 or more students up at anytime without permission as long 
as they stay quiet. (It is possible that this is i r .d J cative of poor 
classroom control. This lax control could prevent time being spent on 
subject matter. ) 

3. Having a well-established routine which minimizes interruptions; room 
runs ''automatical ly". 

Teacher* techniques or methods (positive relationships) 

1. Teacher's ability to admit her own mistakes and laugh at herself or 
use the occasion to teach. 

2. Teacher demonstrates shuwmanship and expressiveness. 

3. Clear explanations. 

A- Having available and using listening centers, aquariums, and other 
"looking" exhibits. 

5. Giving the whole word to a child instead of a hint or unrelated clue 
when he is stuck during reading group. 

6. Using visual aids (picture files, fllmstrlps) for free time activities. 

7. A high percentage of teacher structured time spent in reading groups. 

8. Goes to the chi Id's desk to give help rather than having child come 



up to her desk. 

9. When a child doesn't understand his seatwork, teacher asks another child 
to explain. 

Teacher techniques or methods wieodtive relationships) 

1. Giving a child who is stuck during reading a context clue or definition 
rather than the whole word. 

2. Having either instructional or non-instructional games available In class. 

3. Use of concrete (candy, money) items, special jobs, or classmates clap- 
ping or cheering as rewards. 

4. Teacher's identifying with the class and promoting a ,f we ff feeling as 
opposed to standing aloof and separating herself from the class. 

5. Accepting and integrating student ideas instead of rejecting unsoli- 
cited ideas and sticking with a preplanned format. (This finding is 
borne out to some degree in the behavioral data which shows that for 
Title I, a high proportion of student-initiated comments vs. teacher 
afforded public response opportunities was negatively related to stu- 
dent learning gains.) 

6. Gives complete detailed instructions; prevents errors before they 
happen. 

7. A high percentage of teacher structured time spent in language arts 
and in special activities. 

8. When a child doesn't understand his seatwork, the teacher delays him 
then explains later, if she happens to be busy. 

Student behaviors (positive relationships) 

1. High pupil-pupil interaction which is class relevant. 

2. When students are not working they are playing as opposed to engaging 
in some other activities such as daydreaming or disturbing others. 
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F. Student behaviors (nega+ive relationships) 

I Student obedience and compliance. (Possibly this variable is actually 
a measure of teacher punltlveness which does show a negative relationship 
in some of the other data.) 
2. Student daydreaming or get+ing materials for free time activities when 
not working. 

Correlations in Non-Title I 

A. Classroom management and con+rol (positive relationships) 

1, Punishments for misbehavers involve discussing the incident with the 
child without scolding as opposed to a more physical means of control. 

2. Before starting a lesson cr explanation, teacher says nothing and waits 
for the class to quiet down rather than trying to talk over +he noise 
or signaling with a bell or clicker, 

B. Classroom management and control (negative relationships) 

1. "Busy", cluttered classrooms. 

2. Boring, repetitive, monotonous assignments. 

C. Teaching techniques or methods (positive relationships) 

1. Assigning homework as well as seatwork. 

2. Gaining the whole class* attention before begin r , ing an explanation or 
giving directions. 

3. Having and using science demonstrations or experiments, 

4. Structured time in Math and Art relates to gains in reading and arith- 
metic computation and reasoning. 

D. Teaching techniques or methods (negative relationships) 

1. A high percentage of teacher structured time spent In reading groups. 

2. When a child is stuck during reading, teacher asks another child to 
give him the word. 
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Discussion 

In general, it appears that variables which measure control ever the 
classroom are positively related to effectiveness. The teacher who is well- 
organized, who monitors the class regularly and nips potentially serious prob- 
lems in the bud, and who has well established routines for handling everyday 
procedural matters tends to be mere successful in producing learning gains. 
These data provide strong support for the observations of Kounin (1970). 

It appears that a key factor in effective teaching is organizing the 
classroom environment so that there is maximum opportunity to learn. The daily 
routine and non-academic details are dealt with efficiently and kept to a mini- 
mum so that the task of learning can proceed, and this is especially important 
In the low SES schools. For giving directions or explanations there appears 
to be an optimum level of effectiveness. The teacher who dwells too long on 
details or becomes overly repetitive appears to have a detrimental effect, es- 
pecially In the higher SES schools. 

A few variables show significantly positive relationships in one group 
and significantly negative relationships In the other. For example, for Non- 
Title I children, delaying the child who doesn't understand until later when the 
teacher has time to uevote to him shows a positive relationship with student gain, 
but in Title I this relationship is negative. Also when a child is stuck 
during a reading turn, for Title I, giving i"he child a clue unrelated to the 
meaning or sound of the word is negatively related but in Non-Title I the relation- 
ship is positive. A large amount of teacher structured time in reading groups 
appears as positive for Title I, but negative for Non-Title I. 

O 
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Some cautions need to be stressed in Interpreting these data, which are 
tentative and in need of replication fcr several reasons. First, data were based 
on only 2 to 4 observations per teacher. These frequencies are dangerously smaU 
given the probable day-to-day variation in teacher performance and the effects 
of situational factors such as finishing or beginning a unit, weather conditions, 
etc. Second, the probability data given to indicate the strength of relation- 
ships cannot be taken very seriously because the sample contained only 31 teach- 
ers and over 1000 measures were taken. This obviously violates assumptions un- 
derlying significance tests. Third, partly because of the low number of oppor- 
tunities to observe the teachers, the observers' high-i nf erence checklists and 
ratings show evidence of halo effect and logical error, so that some of the 
variables remain suspect in spite of high inter-observer agreement. Fourth, the 
Pearson r^s reflect only linear relationships and do not take into account pos- 
sible curvilinear relationships between predictors and criteria. We suspect that 
several variables will show such relationships. Fifth, the variances of the 
predictors should be examined, since unusually large or small variances will in- 
crease or minimize the chances for correlation. Sixth, several of the variables 
may be "proxy 0 variables; I.e., they may correlate with another variable which is 
the real predictor with the criterion in much the same way as Income Is correlated 
with education. We are aware that there may be other interpretations of the data 
depending upon one's own persuasion, hence the correlations themselves are pre- 
sented In the tables with a minimum of interpretation. 

Because of the limitations mentioned above, and because, even without these 
limitations, the data represent a unique set of findings which are not directly 
comparable to any other data, replication is being carried out with at least 15 
observations per teacher and with some additional information gained in the form 
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of a personal interview with each teacher this year. This replication study 
will help determine which of the correlates of teacher effectiveness In producing 
learning gains are genuine and dependable. The correlates that replicate will 
then be experimentally manipulated to establish whether or not they are causally 
effective in producing student learning. 

For other data from the first year's analyses, see Brophy and Evertsor 
(1973) and Peck and Veldman (1973). 
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Table I. Correlations between Behavior Observation Scales' and Residua! Gain Scores 
for Total Group, Title I and Non-Title Schools (decimal points omitted). 

2 

Word Word Arithmetic Arithmetic Rater 

Knowledge Discrimination Reading Computation Reasoning Agreement 

Scales _ (N = 3D, (N = 31) (N = 31) (N = 31) (N = 22) > 

Total Total Total Total Total 

Tl NT Tl NT Tl NT Tl NT Tl NT 

1. High Level of Student 

Attention 21 19 32* 25 15 88 

- 02 23 17 10 20 27 17 30 ^33 14 

2. Toacher Often Addresses 

Questions or Problems 

to the Who,, Class 22 ^ 36" J( . « ^-09 _ q? -04 ^ 77 

3. Teacher Is Task 
Oriented, Doesn't 

WastoT.me 27 30» »• IS « .83 

28 21 47 12 17 24 16 14 23 06 

4. Frequent Pup I l-to-Pupi I 
Interaction (Class 

Relevant) - 02 - 22 H 20 16 79 

- 10 00 - 35 - 17 34 07 59** 05 1/ Id 

5. % of Time Teacher 
Lectures or Demon- 

strates 36" }J 40« <( _ 19 2| 13 0J H Q7 89 

6. Negative Affect: 

Criticism, Hostility -tf_ [Q Z \\ - „ -09 8 -15 ^ -2. 14 01 83 

7. Positive Affect: 

Praise, Support II 10 09 16 04 86 

-12 08 28-15 -05 -06 -06 27 -56 06 

8. Requires High Level of 

Genera 1 1 zat Ion, I nf or- 



enco, or Explanation 21 



23 21 -07 - 04 83 



23 | 5 39 04 -24 19 06 -12 - 31 -05 
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9. Hlph Student With- 
drawal, Passivity, or 
Aimless or Repeti- 
tive Behavior -13 -II - 21 -10 -04 75 

25 -18 09 00 20 -19 -08 -10 -51 05 

10. Clarity: Students Show 

Clear Understanding 
of Teacher Presenta- 
tions 20 16 24 19 15 86 

-05 24 17 05 04 23 14 23 60* II 

It. Enthusiasm: Teacher 
Shows Enthusiasm, 
Excitement, Enjoy- 
ment -03 -04 05 01 -09 95 

-20 -13 15 -38 08 -20 -02 -01 -70 -10 

12. Convergent Question- 
ing: Most Questions 
Have Clear-Cut Correct 

Answers 25 19 -02 -05 05 72 

II 28 18 18 16 -10 -16 -01 25 03 



5-Polnt scales rated Intermittently during each visit and averaged across observers. 
2 % agreement with In yl point by two independent raters. 
• p < . 10 
**p < .05 
«*«p< .01 
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Table 2. Correlations between Teacher Behavior Checklist Variables' and Residual Gain 
Scores for Total Group, Title I and Non-Title Schools (decimal points omitted). 



Word Word Arithmetic Arithmetic Rater 
Knowledge Discrimination Reading Computation Reasoning Agreement 
Checklist Variables (N=27-30) (N=27-30) ( N= 27-30 ><N=27-30) <N= 19-21 ) 



Total 
Tl NT 



Total 
Tl NT 



A. Methods of Handling Catch-Up Work 
I. No Remediation; 



Chi Id Skips Missed 

Work -07 
-09 - 

2. Chi Id Must Make Up 

Work but Is Not 
Given Help -03 

3. Teacher Explains 
Work and Has Chi Id 



-09 

-03 - 



07 



25 



07 



Do Part of It. 



26 
16 09 



4. Another Chl Id Is 

Assigned to Help II 

13 - 

5. Chl Id Put In Slower 

Group Temporarily -13 
-27 - 

6. Other -03 

00 - 



18 
24 



04 
13 



-07 



-21 

-33 - 
-17 

-25 - 



B. Rules Regarding Physical Movement 
I. Must Always Get 
Permission to 



Leave Seat 
2. One at a Time 
Without Per- 
mission 



06 
.33 -01 



16 

.16 .29 



35* 



60*** 



36 



Total 
Tl NT 



Total 
Tl NT 



04 



26 



17 

24 03 



15 



31 



- 13 
-18 - 

-02 
07 - 



05 



17 



31* 

32 



II 

20 



-07 -01 

07 - 15 



19 

33 



14 

35 



18 
31 

21 
37 



32 



09 



19 



25 



31 



Total 
Tl NT 



-02 
-II 



-13 



08 



-07 



-19 



-21 



-04 
-21 - 

07 

33 - 



09 

14 12 



27 



28 



Both One None 



I 0 
0 I 



108 



I 5 104 



113 96 



0 6 104 



109 
109 



2 10 97 



0 I 108 
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3. As Many as 4 or 
5 Without Per- 
mission 04 04 -08 -33* -10 0 5 104 

-10 30 04 26 -22 20 -61** - -76 08 

4. Can Go Ouletly to 

Specified Places 
without Permission 



at Any Time 02 



-16 06 05 -03 23 20 66 



-50* 15 -29 -29 -20 06 06 12 47 



5. No Restrictions 06 



05 *" -03 03 01 4 12 93 



42 -03 29 06 29 -10 34 -28 70 -14 



6. Some Chi Idren 
A I lowed Free 
Movement but 
not Others 

7. Only Monitors 
A I lowed Free 
Movement 07 

8. Other 
C. Punishments Used by Teacher 

1 . Stay after 

School 07 
28 

2. Spanklnq 19 

54 

3. Writing Sentences 
on Board 

4. Isolation within 

the classroom 10 
27 

5. Removal from the 

C i ass room -06 
21 

6. Note to Parents 10 
d 7. Send to Prlnclpal-05 
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_25 -18 -22 07 -03 I I 107 

-27 -26 -33 -10 -18 -25 31 -II -21 - 



-03 13 15 - 0 1 106 

26 - 04 - 43 - 26 - - - 

-39** -34* -27 -27 -37* I 3 105 

- 43 -49 -45 -27 -23 -34 -23 -35 - -39 



96 
105 

109 





07 




30* 


16 




13 


5 


8 




22 


01 


78** II 


33 


01 


16 16 


1 






II 




-07 


-05 




02 


3 


09 


30 


10 


-09 09 


07 


01 


14 - 




















0 


0 




10 




12 


01 




-04 


4 


14 


03 


09 


09 


37 -02 


01 


01 


06 -05 








-1 1 




-06 


-09 




-02 


9 


7 


-16 


-04 


-13 


24 -17 


-05 


-II 


21 -07 








12 




1 1 


01 


01 




0 


1 


09 




10 


09 












00 




-II 


00 




01 


3 


4 


-II 


-07 


-02 


-30 -10 


19 


-03 


-O 7 03 







93 

108 
102 



8. Extra Reading, Math, 

Etc. Work -20 -03 

-27 -23 -33 -03 

9. Peer Pressure (e.g. 

"You lost the race 

for your group.") 08 -05 

33 -03 -05 -18 

10. Scolding -09 00 

02 -II -13 24 

11. Discussion of 
Incident (No 
Scolding) 

12. Other 



16 

-25 38 
23 

21 20 
0. Rewards Used by Teacher 



I. Classmates Clap 
or Cheer 



-06 

-41 03 



-41 



00 

21 



-03 



-06 

-40 16 



07 

03 



10 



rn ^ Work Problem on 17 

cKJC 09 

ir " r 1111 Board) 



18 



-05 

-08 -04 
08 

08 01 



-19 



-02 

-08 30 



2. Special Prlvlleges^-08^ ^ 

3. Waiver or Reduction 

of Assignments -35* 

4. Symbols (Stars, 
Smiling Faces, 
etc.) 

5. Token Redeemable 
for other Rewards 00 

6. Concrete (Candy, 
Money, Prizes) 

7. Jobs (Monitor, 
Helper, Clean 
Erasers) 

8. Public Recognition 
(Gets to Read or 



08 

-18 16 
-12 



-31 



-27 



-10 



-19 



-19 



-19 

-59* 17 



08 

-06 22 



II 

-08 15 



19 



-09 15 M 0 5 S04 

18 -14 31 II -21 13 



09 03 07 

-25 08 24 02 18 01 
13 



03 



37* 



-15 



-21 



-13 



16 



I 15 93 
12 23 74 



30 01 37 -07 57 13 



22 02 21 

37 -30 31 -80 44* 
10 



09 



8 14 37 
I 13 95 



22 29 05 12 -41 13 



-02 -23 -29 

-24 02 -31 -19 -8l**-24 

16" 06 

43 -02 26 -09 



-11 



-13 



-29 



-37 



-51** 

-53** 



3 16 90 

0 7 102 

« 

0 I 108 



03 



-16 



10 -24 31 -14 27 -21 



-0! 12 13 

-08 - 15 14 



3 13 93 



2 I 106 



-19 13 10 0 3 106 

-35 -06 08 20 -21 19 



14 00 09 2 5 102 

22 14 -06 05 -85* 21 



26 -II 07 8 16 85 

-26 40 -39 03 ' -26 10 
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9. Other -13 -15 -24 -10 05 0 8 101 

-36 16 -22 (V -31 -06 -29 20 -30 19 

E. Appropriateness of Assignments 

1. Too Short or 

Easy -23 -16 -06 -03 -09 13 !8 77 

-36 -16 -23 -02 18 -II 27 -35 54 -31 

2. Boring, Repitl- 

tive. Monotonous -24 -04 -20 -13 -15 7 20 81 

27 -45* 30 -05 29 -33 08 -33 02 -23 

3. Too Hard: Students 

Can't Get Started 
or Continual ly 

Need Help -II -15 -21 00 -02 3 17 88 

03 -13 -03 -07 -06 -19 02 - -71 10 

4. Continues Acti- 
vity Too Long, 
unt 1 1 They Get 

Boring -27 -22 -24 -10 -05 5 17 86 

-15 -30 -25 -12 -09 -23 16 -26 -20 -04 

5. No Inappropriate 

Assignments 30 22 40** 34* 25 II 17 80 

04 35 08 13 20 40 28 44* -41 33 

F. Olstractlons: What Do Students Do When not Working? 

1. Use Washroom 06 -10 -01 -02 -23 14 18 76 

39 -03 17 -19 22 -04 32 -29 -17 -27 

2. Repeatedly Get 
Suppl les for Free 

Time Activities -05 -16 -05 03 13 6 10 92 

-35 08 -59**11 -21 07 08 -01 05 17 

3. Watch Reading Group 





or other Activity 


-35** 


01 




-34* 


-4|»* 


-33 




3 


20 


85 






-42 -33 


-18 


12 


-27 -35 


-31 -46* 


-21 


-34 








4. 


Talk 


10 


-08 




07 


-13 


-04 




33 


20 


55 


5. 


Play 


00 


-23 




-02 


16 


15 




12 


20 


76 




01 09 


-36 


-L3 


21 - 


39 01 


82** 


13 








6. 


Oay dream 


-16 


-25 




-24 


-15 


03 




II 


19 


78 






-24 -09 


-34 


-II 


-49 -04 


-48* II 


-58 


16 








7. 


Ask for Help or 
























Look More Closely 






















ERIC 


at Work on Board 


03 


-07 




17 


22 


25 




6 


18 


84 






06 


-12 


-0! 


30 17 


45 09 


34 


26 
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0. Disrupt other 

Students -03 -09 -08 00 -04 15 19 74 

08 -08 -12 -08 -01 -13 -03 02 -17 -03 

9 other -19 -17 -36** -19 -20 0 7 10! 

y * 26-38-19 -07 -23 -40 29 -59*** 18 -32 
G. Student Attitudes toward the Teacher 

1. Whon Having Trouble 
Studonts Concentrate 

or Seek Help -J3 -09 02 -08 -17 13 30 62 

-II -19 06 -27 25 -15 13 -24 -12 -20 

2. When Having Trouble, 



Students Merely Copy 
from Neighbor -28 



_37* -21 -10 -06 8 16 81 

-68***- 10 -68***- 10 -14-14 -16-06 -43 - 
3. Studonts Work as 



Well When not 
V/atched as When 

Watched 09 12 25 13 23 II. 16 78 

-26 17 -02 09 -05 31 -07 25 -59 35 

4. Students "Act Up" 

When Unwatched -06 -19 -05 07 07 II 18 76 

04 -05 -36 -01 22 -07 22 -03 43 05 

5. Students Seem 
Amused by 

Teacher -06 04 02 10 06 4 8 93 

Teacner ^uo _ |3 ^ Q2 3Q _ Q? 2Q Q4 Q2 |Q 

6. Students Seem to 
Fear Teacher 10 



Respect Teacher 13 



21 22 21 21 0 5 100 

29 08 19 43* 67***05 31 14 48 21 

7. Students Seem to 

, 6 01 -08 -04 28 12 65 

23 02 43 -17 -29 -01 -30 07 -62 04 
H. Free Time Materials Available (Not Necessarily Used) 

I Oooks -25 -09 -34* -32* -39* 41 16 45 

BOOKS -10 -45* -01 -25 -28 -54** -38 -30 -33 -46* 

2. Learning Centers 

(Any) II 19 15 -05 07 9 21 72 

O -03 06 08 09 17 06 -38 09 -22 12 

ERIC 
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3. Listen! nq Centers -02 -01 

i2 -09 18 -13 

4. Visual (Picture 



Flies, Fi lmstrlps)07 



15 



74**- 1 2 43 -04 



5. Science Demonstra- 
tions or Experi- 
ments 

6. Other Learning 
Centers 



15 

06 14 



18 

16 18 

7. Coloring Pictures 03 

-26 15 

8. Painting, Art 



Oi 

-20 02 



-04 

-13 -02 
-23 

-30 -19 



Activities 

9. Games (Any) 

10. Instructional 
Games 

11. Non- Instruc- 
tional Games 

12. Aquarium, other 



-20 -29 

-12 -25 -22 -35 



-03 

-43 05 



-04 

-30 -C3 



-16 -24 
-56**-06 -45 -21 



-13 -21 
-51* -01 -2i -14 



Looking Exhibits 02 03 
39 -II 34 



-13 



I. Free Time Materials Observed In Use 



1. Books 

2. Learning Centers 
(Any) 



10 
12 06 



20 
-06 20 

3. Listening Centers 09 

04 08 

4. Visual (Picture 

Flies, Fl lmstrlps)40»» 
58***33 

5. Science Demonstra- 



19 

34 04 



II 

16 -03 
-03 

16 -16 
28 

31 20 
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tlons or Experi- 
ments 



31* 

-09 35 



20 

-03 14 



00 -21 -28 19 14 69 

28 -14 -29 -16 -01 -33 

02 -08 -10 4 19 79 

18 -14 31 -25 56 -16 



10 -04 00 5 16 81 

15 -02 07 -14 - - 01 

-02 12 21 I 9 92 

-25 04 -02 19 28 21 „_ , 

|| 08 01 7 27 68 

0! 17 08 07 -13 04 



-II -09 -29 8 19 11 

33 -33 21 -34 -07 -35 m 

-02 -06 -08 14 22 66 

-19 -05 -27 04 -09 -08 



-10 -10 -08 23. 18 61 

-09 -18 -45 14 -54 -02 



-15 -06 — M 24 64 

■13 -16 -37 i3 -21 03 

-15 .-20 00 18 16 68 

-10 -17 -28 -11 -03 01 



-06 -27 -20 25 (6 61 

-0I -2 1 -37 -23 -28 -2 1 



28 02 10 4 15 83 

31 19 -34 12 -36 14 

I9 -07 -28 9 17 76 

60** 02 - -II -04 -3 I 

26 16 13 3 14 85 

16 23 22 13 38 1 1 



29 09 16 0 15 87 

-35 32 -25 21 - 18 



6. Other Learning 

Centers 19 -05 01 12 24 I 10 91 

16 20 -13 -04 -25 08 -02 20 28 24 

7. Coloring Pictures 13 -II 09 00 03 5 23 74 

-24 30 -26 -02 -13 21 -12 08 -47 13 

8. Painting, Art 

Activities -II -16 01 -05 -27 5 15 82 

-02 -18 -04 -27 37 -19 29 -32 -07 -36 

9. Games (Any) -02 -14 01 -06 -08 6 23 73 

-44 II -26 -13 -II 02 -21 03 -12 -08 

10. Instructional 

Games -01 -12 13 03 -10 12 16 74 

-32 08 -06 -19 45 -02 -08 10 -54 -04 

11. Non-Instructional 

Games -06 -16 -03 -05 -06 12 19 71 

-38 07 -26 -II 03 -05 -23 10 -06 -07 

12. Aquariums, Other 

Looking Exhibits II 12 12 -14 -07 4 13 85 

42 02 70***- 10 26 -01 -II 19 06 -08 

J. Use of Peer Tutoring -II -12 -02 -09 -14 7 15 87 

21 -22 23 -31 27 -15 09 -19 04 -i5 
K. Assigns Homework besides Seatwork 

23 13 34» 19 08 4 23 82 

-26 38 -29 25 -04 45* 19 I? -32 13 

L. Teacher Sometimes Underreacts to Control Problems, so Serious Problems Go Unresolved 

-13 -07 -II 03 05 6 13 90 

-35 -06 -52* -21 -12 -12 17 -09 05 06 



I 

Each of two observers completed the checklists twice; scores obtained by summing. 

2 

These are the raw rater agreement data: "Both" means +hat both observers checked 
the Item; "One" means that one checked the Item and one did not; "None" means that 
neither checked the Item. Minor differences In the totals occur because observers 
occasionally felt completely unable to rate on a given Item. 
* p < .10 
9 r ** p < .05 



24 

Table 3. Correlations between High-Inference Ratings and Residual Gain Scores for 
Total Group, Title I and Non-Titie Schools (decimal points omitted). 



Word Word Arithmetic Arithmetic Rater 

Knowledge Discrimination Reading Computation Reasoning Agreement 



Ratinas Variables (N=24-28) (N=24-28) 



Total 
Tl NT 



26 

-21 42 



14 

-52 44 



-17 
I i 



I . Typ ical Af f ec- 
tionateness 
Level 

2a. Most Intense 
Affection Ex- 
pression Ob- 
served 

2b. Most Intense Neg- 
ative Affect Ob- 
served 

3. Solidarity with 
Class: Teacher 
Identifies, Pro- 
motes "We" Feel- 
ing 

4. Patient and Sup- 
portive When Cor- 
rect I ng 

5. Students Allowed 
Choice In Assign- 
ments 

6. Accepts Student 
Ideas and/or Inte- 



rnfp grates them Into 04 
£j\|^ -16 08 

Discussion 



22 

-25 39 



20 

-14 32 



-01 
16 



-14 



Total 
Tl NT 



14 

-01 19 



01 

-40 16 



-18 

37 



00 



16 

15 



08 



04 
II 



-15 



00 
31 



-22 



06 
18 



-13 



(N=24-28)(N=24-28) (N= 18-20) 



Total 
Tl NT 



14 

-25 26 



22 
-04 24 



-06 
-13 08 



16 

-05 14 



10 

-25 14 



-06 
33 -33 



09 

06 -04 



Total 
Tl NT 



21 

-03 35 



28 
20 



30 



-06 

-28 42 



01 

-34 28 



-03 

-28 15 



-20 

-22 -26 



-13 

-20 -16 



Total 
Tl NT 



1 1 

-49 17 



14 

-53 22 



-15 

-61 24 



06 

-83**20 



03 
-51 



-08 



-24 

21 



-37 



-21 

-84* -28 



86 



90 



75 



84 



86 



82 



79 



25 



-17 



25 47* 



-03 
-33 01 



7. Adn.lts Own Mis- 
takes; Laughs at 
Self or Uses Oc- 
casion to Teach 

or Motivate 04 
44 

8. Usual ly Bends Close, 
Gets Down to Ch I Id's 
Level 

9. Goes to Seats to 
Check Work; Doesn't 
Stay at Desk 

10. Usual ly Speaks to 
Individuals rather 

than Whole Class -13 

-13 -22 

11. Uses Advance Organ- 
izers in Introducing 
Activities 

12. Gives Complete, 
Detai led Instruc- 
tions; Prevents 
Errors before they 

Happen 32* 

01 36 

13. Students Eager to 

Respond; No Fear 17 

27 03 

14. Teacher Waits Pa- 
tiently if Student 
Doesn't Respond 



30 
-01 39 



ERLC 



Promptly 



13 

21 06 



12 

78***-30 



28 

46 1 1 



-12 

- 1 2 -20 



-04 

-20 -01 



19 

14 15 



20 

33 02 



24 

32 II 



02 
44 -31 



-09 

-02 -23 



20 
■10 30 



-26 

-34 -22 



-04 

-35 24 



-19 

34 -30 



14 

-31 24 



06 14 
II -04 35 -12 



02 

-26 02 



II 03 
-37 22 -23 18 



02 

-73 13 



84 



71 



22 -13 -14 83 

21 16 -06 -25 -53 -14 



84 



71 



12 -17 04 67 

-44 18 -72***15 -59 10 

27 19 02 86 

14 20 15 15 07 -02 



-01 -15 -13 73 

-25 01 -34 -03 -67 -13 



15. Non-Competitive Atmo- 
sphere; No Signs of 
Eagerness to See 

Others Fai I -12 

13 -13 

16. Students A I lowed to 

Work in Cooperative 

Groups 1 0 

-34 26 

17. Teacher Recognizes 

Good Thinking Even 
When It Doesn't Leod 
to "Right" Answers 

II 

14 02 

18. Democratic Leader- 
ship Style: Students 
Share in Planning 
and Decision Making 

20 

02 16 

19. Few Restrictions on 

Students During 

Seatwork Periods 10 

-02 34 

20. Students Expected 

to Care for Needs 

Without Getting 

Permission 38** 

22 41 

21. Teacher Concerned 
with Substantive \ 
Content, not Form, • 
of Student Responses 




08 



51 -10 



-20 -19 -06 -17 50 

01 -24 03 -10 16 -22 II -27 



-07 15 00 17 71 

-20 00 27 14 -12 10 -47 24 



-07 03 -19 -22 83 

29 -41 -04-11 -26 -23 -46 -26 



09 25 -07 -17 81 

26 -16 07 14 -03 -23 -63 -25 



-06 08 08 01 69 

02 00 19 21 28 -05 30 -02 



15 29 17 26 66 

29 -01 26 21 13 15 57 22 



-06- -10 23 14 75 

33 -31 08 -15 47 -01 62 05 



ERIC 



22. Teacher Stresses 
Factual Real Ism, Re- 
jects or Corrects 

Chi Idish Idealism 06 

26 -04 

23. Teacher Credlbl I Ity: 

Students Seem to 

Believe and Respect 

Teacher 20 

-08 27 

24. Showmanship: Teacher 

1 5 Melodramatic, Ex- 
pressive, Gushy, 

Emot I ve - 1 1 

-18 -15 

25. Teacher Gets Atten- 
tion before Start- 
ing, Doesn't Try to 

Talk over Din. 30 

17 33 

26. Chaotic, Unplanned, 

Poorly Scheduled -25 

-33 -22 

27. Teacher Seems Con- 
fident, Self-Assured 

22 
42 14 

28. Politeness: Teacher 

Regularly Says "Please," 

"Thank You," etc. 02 

02 -03 

29. High Concern about' 



Achievement 
30. Room Is Attrac- 
tive 



23 
16 19 



44* 

36 45* 



-07 

40 -28 



II 

17 -01 



09 

-15 15 



27 

41 10 



-38»« 

-48 -33 



II 

44 -07 



00 

29 -26 



15 

31 -06 



35* 

55* 14 



-10 

00 -20 



32* 
-05 45* 



-20 
12 -42 



-17 

-40 -04 



-16 

-36 -19 



16 09 
-16 19 -17 27 



04 
-61 



-01 

-30 20 



. 05 

46 -48* 



00 

-66 07 



-24 

20 49* 



02 00 
II -12 -15 12 



09 -05 
-24 07 -31 08 



32* 04 ! ! 

34 17 -10 05 08 10 



86 



76 



-07 24 04 80 

30 -30 58** -02 19 02 



82 



78 



02 06 -04 90 

12 07 05 06 03 -07 



-02 98 

-43 02 

-07 88 

-61 00 



74 



28 

31. Teacher Gfves Much 
Encouragement to 

Students 05 05 II -07 -08 88 

-16 04 24 -28 06 -09 -26 03 -52 -08 

32. Room Is Uncrowded 

56*** 40** 53*** 46*** 43** 74 

43 59** 21 41 03 62*** 40 50** 60 48* 

33. Teacher explains 
Rules or Decisions 
When Reasons Aren't 

Obvious 40** 30 19 02 06 87 

32 41 53* 07 -08 18 -25 16 -64 10 

34. Teacher Wei I Or- 
gan I zed. Prepared 46*** 32* 21 -03 14 84 

50 42 55* 09 -21 25 -47 27 -36 25 

35. Teacher Regularly 

Monitors Class, 
Knows What's Go I ng 

on 35* 32* 23 00 00 86 " 

34 31 53* 08 -04 20 -25 14 -40 04 

36. Smooth, Efficient 
Transitions, Little 

Time Wasted 49*** 36** 35* 13 14 70 

50 47* 64** 15 08 37 -08 25 -24 19 

37. Monitors Determined 
"Automatically" ' 
by a Systematic 

Procedure 37* 37* 50*** 27 18 74 

41 29 56** 16 41 44* 12 33 36 16 

38. "Busy," Cluttered 

Classroom -02 05 12 -20 -09 86 

-26 - 03 -07 22 -13 16 -43* 10 -25 

39. Students Compliant, 

Obedient 27 23 23 12 09 94 

-04 36 24 16 -19 33 -21 34 -84**20 

40. Teacher Gives Overly 

Explicit, Repetl- - 

O 

ERJC +,v « Directions -12 00 -23 -53*** -37* 74 

asa ^ m -07 -2 i 13 -13 03 -51** -62**-47* -50 -45 



29 



41. Well Established 
Routines Minimize 
Interruptions; Room 
Runs "Automatically" 



29 22 27 06 08 82 

-01 34 32 07 -01 26 -27 25 -83**17 



I 

First three scales have 13 points; all others have 5. Scores obtained by sum- 
ming across two raters. 
2 

% Agreement across rater pairs; within two points on first three scales, within 
one point on all others. 
* p < .10 
** p < .05 
*** p < .01 
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Table 4. Correlations between Percentage Estimate Variablesl and Residua! Gain Scores 
for Total Group, Title I and Non-Title Schools (decimal points omitted). 

Word Word DIs- Arlth. AHth. Rater 2 

Know- crlmlna- Read- Computa- Reason- Agree- 
ledge Hon Ing tlon ing ment 

Percentage Estimate Variables (N=22-3I ) (N=22-3I ) ( N=22-3I ) (N=22-3i ) (N=l5-22) 

Total Total Total Total Total BON 



Tl NT Tl NT Tl NT Tl NT Tl NT ? 8 n 

n e 



A. Time Uti lization 3 

1. % Total Time Structured by 
Teacher -09 09 -30* -21 -16 

2. Structured Time In Lang- 11 - 19 18 03 " I0 " 43 * "° 9 - 30 "° 2 "20 
uage Arts -04 -08 02 03 17 

3. % Structured Time in " 50 * 14 ~ 34 07 " 26 17 ~ 15 15 "06 21 
Math 24 05 36 32 25 • 

4. % Structured Time In 13 25 " 10 " 08 21 34 25 44 * 36 29 
Art 30* 12 43** 25 45** - 

5. % Structured Time In Spell- "° 4 39 " 25 24 "° 9 58 ** 07 34 42 4 ** 

•ng -01 10 -10 -13 -21 - 

6. % Structured Time In Read- 09 " 09 "° 815 " 21 17 ~ 36 01 - 30 
Ing Groups -16 -09 -22 -05 -33 

7. % Structured Time in 38 " 33 54 * ~ 34 39 " 40 * 52 *" 42 * 39 " 43 * 
Social Studies -05 09 04 01 01 

8. % Structured Time in Tran- "° 5 " 10 10 01 26 "° 9 ~ 43 14 " 59 ^ 
sltlons 09 03 -05 -01 12 - 

9. % Structured Time in "° 2 16 "° 8 13 " 24 06 ~ 15 11 29 « 
Morning Routine -09 -14 -03 02 -17 - 

10. % Structured Time In 02 ~ 13 " 26 "° 8 13 "° 9 32 " 19 28 " 27 

Special Activities -09 -03 -25 -29 -12 

-03 -08 02 04 -19 -23 -59**02 -48 -06 

B. Methods Used to Call for Attention 
I. Says Nothing, Walts for 





Quiet 


29 


28 


* 

31 


01 


07 


21 


16 


15 


2. 


Raps Desk Lightly, Uses 


-05 ?7 


22 26 


-27 45* 


-45 17 


-77 07 










Normal Voice 


-13 


-09 


06 


06 


01 


1 


16 


35 


3. 


Gimmick (light flick, bell, 


-35 31 


-07 26 


27 13 


16 26 


-05 25 










cl Icker) 


13 


-06 


08 


26 


18 


5 


8 


39 


4. 


Raises Voice Over the 


09 -36 


-14 -22 


06 -46 


34 -34 


33 -19 










Din 


-26 


-14 


-29 


-18 


-15 


22 


22 


8 






-25 -06 


-32 -14 


-17 -II 


07 12 


68 10 









31 



5. 


Raises Voice and Singles 


























Out Individuals 


-01 


03 




-02 




-10 




-03 


12 


23 


!7 


6. 


Shouts, Becomes Angry, or 


-03 -07 


-07 -03 


-12 


-02 


-17 


-12 


38 


-12 










Scolds Class 


-10 


-16 




■18 




08 




-01 


5 


5 


42 


7. 


Shouts, Becomes Angry, or 


42 -15 


06 -02 


45 


-15 


39 


-09 


32 


— 










<jwvJ l vjo I iiu 1 V 1 vJ'JC 1 o 


-34 














-fift 


j 


p. 


*T 1 


o 

o • 


Whispers or Speaks Softly to 


40 -30 


04 -13 


23 


-27 


36 


-16 


20 


-07 










Nearby Pupils (at first) 


04 


07 




06 




23 




00 


0 


5 


47 


9. 


Other (includes any method 


31 15 


38 27 


29 


06 


21 


19 




12 










not listed above) 


01 


-20 




•18 




-04 




Oi 


3 


12 


37 






■03 12 


-19 -10 


41 
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C. 
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Estimated % of Students Paying Attention 
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D. What Does the Teacher Do When a Child Doesn't Understand? 
I 



Stops What She's Doing, 
Explains 

2. Delays Child then Explains 

Later 

3. Delays, but then Fails to 
Foi low Up 

4. Asks Another Chi Id to 
Exp la i n 

5. Scolds Chi Id for Not 
Understand! ng 

6. Encourages Child but Give 
No Help 

7. Refuses Help ("You're on 
your own.") 

8. Sends Child to Aide or 
other Adult 

9. Other (includes any method 
not listed above.) 
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E. Teacher Goes to Child's Desk to Give Help, Doesn't Stay at Desk 
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1. Gives Word 

2. Gives First Sound or 
Syllable 

3. Child Starts Sentence or 
Paragraph Over 

4. Gives Context Clue or 
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Reading Group 
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Asks Another Child to 
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or Meaning ("It's one of 






















our new words.") 
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Tells Chi Id to Skip, Go 
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to Next Word 


-28 




•03 


-11 




13 


-07 0 


2 


27 


8. 


Other (Includes any method 
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'two observers estimated percentages for each appropriate category; scores obtained 

by averaging. 
2 

These are the raw rater agreement data: "Both" means that both observers entered 

a % >0 on the Item; "O.ie" means that one entered a % > 0 and the other entered "0;" 

"None" means that both observers entered "0." Differences in totals occur because observers 

were not always able to estimate with any confidence. 
3 

Time utilization data were computed from Information on the classroom coding sheets 
concerning the starting and ending of activities. Reliability coefficients were not com- 
puted here because agreement was near-perfect; minor differences were handled by aver- 
aging across observers. 
4 

Pearson £ was computed for this variable, since both observers made % estimates In 
every case; £ = .67. 
5 r = .50 
* P <.10 
** . P < .05 
*** p ^ .01 
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